BI topics: session on inflation, systems change and healthcare access

Good morning, everyone. My name's Andrea Willis. I'm BETA’s acting managing director at the moment. Welcome to the first of three sessions of BI Connect 2023, the series we'll be exploring leading work across across the behaving industry. This morning he is each of our speakers. And on behalf of all of the presenters here today. I'd like to acknowledge the Ngunnawal people as traditional custodians of the ACT, that's where we're hosting from today and recognise any other people or family with connections to the lands of the ACT and region.
We acknowledge and respect their continuing culture and the contribution they made to the life of this city and this region. I would also like to acknowledge and welcome other Aboriginal and Torres Strait Islander people who may be attending today's event. Thank you so much for joining us here today. The very brief background for those of you who aren't already aware, BETA is the Australian Government's behavioural and social science research unit.
We apply behavioural insights to public policy. We operate from within the Department of the Prime Minister and Cabinet and our mission is to improve the lives of Australians by generating and applying evidence and overall and social sciences to find solutions to complex problems. A core part of our mission involves building capability and this event is one of the many initiatives that we run to share knowledge and build awareness of behavioural insights and the role that it can have in supporting the development of government services policies and programs.
If after today's event you're feeling inspired. Please visit our website where you can see further information about BETA’s projects as well as the various tools and resources we have that can help you learn more about them, about applying behavioural insights to a project of your own. So onto today's session, you'll hear from a number of researchers and practitioners working across different domains on a variety of different topic.
First up, we have Professor Nick Biddle from the ANU and he will present his research findings on the effects of priming on inflation expectations. As you're well aware, the current economic environment is focusing media attention and public commentary on financial indicators and future mood. Nick has been looking into how priming affects the distribution of inflation expectations, and we'll have some reflections on the characteristics of individuals who are most susceptible to the effects of priming.
Following Nick, our conversation will shift away from finance was health care. Alex Galassi and Dr Kate Reid will be presenting a project from the New South Wales Behavioural Insights Unit, which tackled sludge to increase the bookings and attendance at critically important health and development checks on children. And finally, our last presenter, Professor Liam Smith from BehaviourWorks Australia is going to get you all thinking about ways that behavioural science could be and become more sophisticated in its ability to offer solutions for public policy dilemmas.
At the end of all of these three presentations, we'll get all of the presenters together for a panel style Q&A. But please, if you have any questions while the presentations are focussed, you're welcome to submit these during the talk by the Q&A function. So I think it's time to get into it. First up, I'd like to welcome our first speaker, Professor Nick Biddle from the Australian National University.

Nick is the Associate Director for the ANU Centre for Social Research and Methods and he's director of the Policy Experiments Lab. Nick has a Bachelor of Economics from the University of Sydney and a Master of Education from Monash University. He also has a Ph.D. in public policy from the ANU. And while it's not his most important claim to fame, Nick is also a valued member of his own academic advisory panel.

Thanks, Nick, and over to you.

Thanks for the introduction and I beg to differ. It is my most important claim to fame, and at least it is. It is in the top five as my claim to fame claims to fame. It is a pretty important part of what we do and a really important link for us. So just confirming that you can see the slides.

So after you've gone, it's perfect. Okay, great. Sorry. Yeah. It looks as mentioned, what we're going to talk about is the work which we've been doing using our survey data focussed on understanding priming on inflation expectations. So this is a collaboration between myself and Dinith. Dinith Marasinghe is a researcher at the ANU Centre for Social Research and Methods and has trained in economics from University of New South Wales as well as the ANU.

So in our view, I'll get into an introduction in a second, but is essentially three kind of main things we want you to take away from this talk. One is the main findings around understanding, priming and priming. It is an important part of behavioural insights in the way in which people kind of form their views and make decisions.

Second is, is the specific topic around inflation expectations and really just an understanding that people's inflation expectations are highly contingent and they're influenced by the information which they have given the importance of expectations on decision making and how that decision affects ongoing inflation. I think that's quite important in of itself. And finally, really just wanted to highlight the use of survey data to and high quality survey data to understand and explore these issues, all of which are all kind of go into a bit more detail and just I'm not quite sure of the mechanism, but we're very happy to for these slides to be shared and for people to have access to them and have

a follow up conversation. Okay. So let's get into the the kind of the rationale. So central banks or lots of researchers really do spend a lot of time understanding inflation expectations. And the reason why is that people's decisions based on the future so they make decisions about their own kind of wage demands and businesses make decisions about price sitting on with an expectation about kind of future prices.

So these expectations did directly contribute to forecasts created by banks, all of central banks, as well as kind of commercial banks, all of which impact on policy decisions. So essentially, subjective inflation expectations can affect kind of wage negotiations and decision making by individuals, but they also affect kind of consumption decisions by households. So if you think prices are going to go up into the future, you're going to make a very different decision than if you think prices are stable.

So prices are important, but we actually don't know a lot about how to design a kind of inflation expectation surveys is a very tricky thing to to ask about the reality is that people don't really have a good sense of what inflation is, the current point in time or what it will be in the future. There's a lot of uncertainty and there's a lot of variation by who's surveyed, how the question is asked, and importantly, from the point of view of this paper, what information is given to individuals when asking about their inflation expectation.

So that's kind of the priming aspect of it. So we don't when we're doing surveys, we don't want people to answer in a complete vacuum. You need to have some information. But at the same time, we also don't want to lead people into kind of giving an answer, which is just driven by the information which they given. And finally, it's important in terms of understanding inflation, whether we focus on point prediction.

So an average or whether we look at the distribution and what we know from our data and other data is that there is a very wide distribution in in inflation expectations and that people can not only move around so you can not only move around averages, but you can also change the shape of the distribution. So what we look at is, is a very simple priming, which is providing historical inflation expectation at the beginning of the survey question, and that's been done before and other in other data collections.

But importantly, we we use different different treatment groups across our longitudinal sample, but also have quite detailed large, large samples. So we can look at not only what's the average effect, but also how it varies across the distribution of individuals. Okay. So there's a there's a literature around priming and the way people make decisions, and it's kind of the bread and butter of of kind of behavioural insights.

But with regards to inflation expectations, there's a, there's a couple of kind of key, key insights, I guess, that forecast are based on what's held in front of mind in people's memories and also that extremes are more salient than things which are a kind of the around the kind of middle of the distribution. And so that means that individuals who whose job is not to kind of forecast inflation or that they're not kind of reading the Financial Review every morning they're there in their expectations are likely to be impacted by extreme price changes which are front of mind.

So they might see a very large increase in a particular good or service that's front of mind. What they don't see is is a smaller price changes or or drops in prices. So what that means is, is that people tend to overestimate what inflation's going to be into the future. But when some information is provided on that, it gives an anchor to individuals to to to then make kind of future decisions based on.

And and the expectation is that giving that external information will lead to lower inflation expectations than would otherwise be the case. So as mentioned, there's been a few surveys which have looked at the effect of priming or providing or specific for priming, which is providing historical information, which kind of backs up some of these findings so new and have found that indeed it did lower inflation expectation.

Um, but also read and others found that it not only affected the point estimate but also the shape of the distribution. And then remember the shape of the issue. Distribution matters because for a given average, the more outliers there are, that's going to drive people's behaviour in quite different way. And the limitation of those studies though, is that they were based on kind of aggregate level data and did not allow us to look at a kind of how individual characteristics are affected or influenced by that kind of that shape of the sorry that individual characteristics were related to the effect of priming.

And as mentioned, there's some broad literature around kind of anchoring effects, which which will kind of expand on. Okay. So just a hypothesis, the there might be some who for whom anchoring has less of an effect. So those who who who have some kind of extreme price experience have extreme price changes, which therefore that's going to stay front of mind, or those who are less sensitive to external information because of some form of expertise.

So what we do is we look at the effect of priming broadly and examine how individuals respond at different points of the distribution, what types of individuals are more susceptible to priming, and then how priming affects uncertainty. So that contribution, that's how motivation, where do we get our data from? We get our data from a new poll, which is a a roughly quarterly survey survey run by the Social Research Centre - spelling mistake.

The based on the Life in Australia panel, it's nationally representative and it includes importantly longitudinal tracking of individuals. So we know essentially what someone's inflation expectations were in the past as well as what treatment group they were in. When we assign different forms of it priming information as the as the year progresses. So this is I sorry, 20, 23, January, April and August 20, 23.

So we have we ask inflation expectations in January and then we have to essentially I ways where we provide to a treatment group. The historic inflation data is a form of priming. So we have of those who answered all those questions, we have a sample size a little over 2000, which is larger as far as we can tell, than other surveys, which have kind of focussed on inflation expectations and particularly which is focussed on the effects of priming.

So here's how his our treatment which was presented to independent random samples. So remember, as a new poll, 55 was in April, 56 was in August and you poll 54 was in January where we didn't everyone was in the control group in January. So for those in April, we gave them the, the CPI as of the prior quarter.

And then we asked them a range of questions about the inflation expectations. So we not only we asked whether they expected prices to go up or down and also the level and for everyone, we we gave a little bit of information, which is that zero means no price change means a doubling and a -50 means that prices would go down by half.

So we give a little bit of background information, our arithmetic information, to to to try and translate their expectations into numerical value. So just to remember, reinforce, in January, everyone was asked just the inflation expectation question. In April, there was the priming treatment for 25% of the sample in August, another 25% of the sample could have been the same people.

It was done independently within primed with a different with with the current inflation as of the 12 months leading up to two August. So that's this information here. We're keeping in mind that inflation moderate a little bit between April and August. Okay. So what do we find lined up very well with our kind of priors, which is that there's quite a high level of inflation expectations, so much higher than any observed inflation over the period.

And this is kind of consistent across inflation expectations data. So the control group expected prices to go up by a little over 20%. And the treatment group, which received that priming, expected inflation to go up by a little bit less, but still far higher than any kind of historic price data. So people don't have a very good idea of what prices are going to change in in the future.

People overestimate inflation, but we do say it means the treatment group. There was a smallish treatment effect. So giving that information did reduce inflation expectations. And you can see here the distributions. And not only did the the treatment group, but the control group also shifted a little bit to the left. So there is still even with the the priming, even telling people that over the last 12 months, prices have only gone up by a little over 6% or seven or so percent in the in in the April survey, there's still a large number of people who have inflation expectations very far to the right of that distribution.

So this is the summary. Lower the mean and median and there was some differences. Also the the there was a high standard deviation in the treatment group, which is a little unexpected, but it was partly to do with to do with the fact that the priming had very little effect on those at the extremes of the distribution. So I'd move people in the middle to the left, but those are the extremes didn't change.

So how do we look at how do we model the effects? Essentially what we do is we run a quantile regression where what that does is quote control. That regression does is it gives you a different parameter. So the treatment effect across different points of the distribution of the dependent variable. So essentially what quota regression allows us to do is to say, okay, for those who would have been at this point on the distribution, what's the effect of the treatment, those who are at a different point of the distribution, what's the effect of the treatment?

And then you can see kind of how priming effects people differently depending on where they would have been on that that distribution if they were in the control group. So what we do is we include a set of controls as well to take into account a kind of background characteristics which might affect inflation expectations, hold those constant and then really focus on the treatment effect, which was the effect of priming.

And so these are essentially it's a fairly there's quite a fair bit of information in this in this chart, but the main kind of point is so the the black bar is so we just focus on the on the left, the similar pictures for, for both for both years. Essentially the black bar is the average treatment effect. So that's the average effect of priming across everyone in the sample, the the dotted lines are then the confidence intervals around that average effect are zero is essentially if there was no treatment effect.

So essentially this priming had no effect at all on people's expectations. And then the line, the blue line is then the estimated treatment effect at each point on that distribution across people's inflation expectations. And so the main point is that the further away you get, the more unrealistic your inflation expectations are, the greater the impact of giving giving those people information about historic inflation until the very extremes of the distribution where essentially the for those at the extremes, the priming effect becomes not significant and quite quite a lot of uncertainty at the end of that distribution.

But essentially for the main part of that distribution, the further away you are from realistic inflation expectations, the greater the effects of priming. That's essentially what the quantile regression shows. And you can see this here across different points of of the distribution. So the 10th, the 25th, this the median, the 75th and the 90th percentile distribution. So essentially, as I said further, you go to the right, the greater the effect of priming up until up until a point.

And this is essentially just summarising it, summarising. So the effects of priming lowered inflation expectations for those who are more rational, those who had less rational values of inflation expectation, the lower the effect of priming. So some demographic information females had high level inflation expectations of if differences across age and then some important differences by income as well.

So just conscious of time. So I'll just go through this part reasonably quickly. We looked at who was more, uh, who was more impacted by or the individual characteristics which were associated with, with priming, having an effect. So essentially we use linear regression where we look at the difference between the anchored value and the true value, and then look at the effects of different characteristics on, on, on kind of the effect of of that of that priming.

Okay, So essentially this is looking at treatment of the heterogeneous treatment effects. So females had a higher difference in inflation expectations relative to males. So this is essentially a different kind of treatment effect. Those who are in older age group had a higher and at a higher difference. Lower levels of education were more likely to shift their expectation and those in the middle and the top of the in distribution were were more likely to to kind of reverse their expectations.

And the final point is that we also includes historic priming. So people who who had been primed in a previous survey and found that there was there was no effect of historic priming. So essentially the the the your historic experience in the survey had no effect on the effect of the priming in future surveys. Those who had who were further away from the main in previous waves were less likely to move closer to the the anchor than when primed in a in a future survey again.

So lots of sensitivity analysis here which I'll I'll skip through and just touch a little bit last couple of slides around uncertainty which is essentially looking at the uncertainty as measured by giving round numbers. So if you just to be really make up some numbers, if you give a value of 6.75, then you probably have more certainty around your estimate than if you give you a value of ten.

To give you a value of 13, it's it's likely to be using more information that if you give a round number of 15 or 20. So there's a lot of data around kind of using round numbers as a measure of uncertainty. And there's some survey evidence which shows that the proportion of people who give round numbers in inflation expectations surveys is more likely to increase during periods of economic uncertainty.

So we make use of that data by essentially looking at whether the being in the treatment group either reduces the proportion of people who who have uncertainty around inflation expectations or whether it increases the percent of people who give a non round number. And we can see this data. There's there's the modelling behind it. But this data essentially shows that not only does priming shift the distribution, it shifts the average.

It also reduces the amount of uncertainty. So fewer people giving round numbers when given some background information about historic inflation. And I'll as I said, there's modelling behind this, but given time I'll just go on to the key findings. Okay. So priming had an increasing negative effect on level of inflation. So inflation, so priming had had an effect.

It became a stronger the further people were away from that historic data. But beyond that, essentially people who are in the 80 to 100 percentile, if the priming had had little or no effect, essentially showing that those are the very extremes of the distribution, no matter what information you give, it's less like it's unlikely to bring them back to a rational kind of expectation of inflation.

Second result is that the effects were not homogeneous, and there's some characteristics which were associated with the size of the effect. A third finding is that the the priming not only reduced kind of expectations, but also reduced the level of uncertainty. And in just to kind of put it all together, providing kind of historic information when trying to elucidate inflation expectations can help, but you still have quite quite an extreme quite a number of people at the extremes of the distribution.

Still quite a lot of people around kind of giving uncertain values and essentially a lot of uncertainty around inflation expectations. Priming helps, but making economic and policy decisions around kind of consumer inflation expectations should be done with quite a fair degree of caution. Still a lot of uncertainty, still a lot of values, which a far to the to the right of what you'd expect place based on historic inflation data.

And I will finish there.

Thanks so much, Nick. That was excellent. I will now move right on to our next speakers. We have Alex Galassi and Dr Kate Reid from the Behavioural Insights Unit within the Department of Customer Service from our colleagues over at New South Wales Government. Alexi is a senior behavioural adviser in the New South Wales Behavioural Insights Unit. She's driven projects to improve customer outcomes through behavioural science, partnering with teams from across government to quantify, identify and reduce unnecessary frictions to the customer experience.

Alex holds qualifications in rehabilitation, counselling and psychology. Today Alex is joined by Dr Kate Reid, the programme manager for Life Journeys and the Department of Customer Service lead for the cross-government Brighter Beginnings Initiative to improve the developmental outcomes of New South Wales children. Together they will present on work conducted with the Ministry of Health and the Illawarra Shoalhaven Local Health District to identify and I'm sorry to identify and overcome frictions that were making it harder for families to complete their health checks.

Alex and Kate have some really impressive results to share from a recent pilot with us today. I'll pass over to you now, Alex and Kate, thanks.

Thanks, Andrea. And it's very exciting to be here. Thank you for having us. Kate and I are very excited to be able to share some of the work that that we've done and and how sludge audits have contributed to that work. So getting started, what I'll be going through firstly is just a bit of an overview of sludge and our study audit method and really focusing on how sludge can present barriers to people accessing preventative health services will then go through our really exciting case study with with case and then on to some resources that you can use to bust sludge in your work.

So firstly, just a very quick definition of sludge, and I'm sure many of you attending today already know about it, But essentially what it is, is those unnecessary frictions that make it harder for people to access services. And what we're trying to do in New South Wales is create sludge free government services so people have greater access to services and there's greater trust in government as well.

But of course, we know that sludge does exist in government and this makes it harder for people to achieve their goals and means that there is not fair and equal access to these services. So when we think about sludge in government, it's information that's difficult to find not written in plain English, confusing and onerous eligibility requirements where people don't know whether or not they're eligible or could access a service.

Excessive wait times, particularly where people aren't kept up to date on the status of what they're waiting for, and also the psychological factors and the systemic issues that can cause exclusion, anxiety and distrust and what we know is that it is often the people who would most benefit from government services, who are the ones who are subject to this exclusion most often now looking closely at health behaviours, we know that there is an intention action gap, so we can intend to follow through with these important preventative health behaviours.

But there is this, there is this gap and we know that sludge can make that gap larger. So sludge or extra steps can, can present a barrier to action. So we think about things like ineffective booking services, limited or confusing online information, complicated paperwork, etc. All of these barriers, all of this sludge can stop people from accessing these really important health services and undertaking these really important preventative health behaviours like cancer screening, exercise, skin checks, etc..

And this is what our case study today will look. But before we jump into it, I'd like to take you through what New South Wales is doing more generally with our sludge audit method. So in New South Wales we've developed this method for identifying, measuring and then reducing sludge, and we've piloted this method initially in 2020 and now it's being taken up and used more widely.

The case study that we'll go through in a moment was one that was a sludge audit that was done last year through one of our sludge-a-thons. And through all of these sludge audits, we followed this seven step process. So first is a behavioural journey map, which is where we step out. Every single thing that a customer or a citizen may do to access a certain service.

So from the point at which they decide to to the point at which they successfully access it, or in some cases where they may drop off, we then gather information. So we gather inputs and data to help us understand what that journey looks like. So how long things are taking and what experiences like that data then helps us complete the sludge audit itself, which is estimating the time and cost of each of the steps in that process, assessing the customer experience, using our sludge scales and looking at any barriers to access and equity without access and equity checks.

So that gives us full metrics of time, cost, effort and inclusion. We then look at the results, so we analysed the results of that. That audit to prioritise the sludge that is most impacting people's access to a service. With those with that sludge prioritised, we can build solutions that are going to be most effective to helping people achieve their goals.

So that's our seven step method. As I said, we've we've used this method and, and used used our study audit tool to conduct many sludge audits. At this stage there are over 50 that have been completed. But we'll take you through a case study. Now, which I think is one of our most successful stage audits. And you'll see you'll see that in the in the impacts that Kate will present in a little bit.

And what this study audit was, was part of, as I said, last year's budget on where we partnered with Life Journeys, which is which is where Kate works, which is part of the customer experience unit in the Department of Customer Service and New South Wales Health. So it's a really great collaboration between the Department of Customer Service and and New South Wales Health and what what the team focussed on was these milestones, child health checks.

So in New South Wales, children have access to milestone child health checks, which are free, and that's up to the age of five years old, but they're underutilised. So in that when we look at the stats we say that 90% of carers and children take up that first check, but then when it gets to the age of four, they're only around 10% or less than 10% of, of carers and children who are accessing that child health check.

And you can imagine, you know, the kind of barriers that, that and the cognitive scarcity and overload that that families are experiencing in those early months of and weeks of, of their children's lives. So you can say then how sludge and the barriers can, can prevent people from following through. So what we wanted to do with this sludge audit and what Kate and the team wanted to do was prioritise the sludge so that they could develop solutions to address it.

So I'll take you through now the steps of the sludge audit and what they found when they completed it. So first, looking at the behavioural journey map, you can see here that the team mapped out all of the behaviours are that a carer undertakes to get their child to a child health check. And you can say that there were around that were 38 behaviours and we've grouped those behaviours into eight different phases to kind of show that the key steps that someone might take.

You can also say in those blue boxes that there are different paths in the customer journey. And what that shows is that some people may have greater difficulties or might be presented with more sludge when they when they when they go through this journey. And that was really important to say later. We then looked at the time the time it takes.

So we completed the time audit and you can see here that that the time for each of the phases we found that trying to book an appointment could take around 60 minutes of active time. So that's active time spent looking up the booking details and calling. And then on top of there could be up to nine days of waiting.

So nine days waiting for the clinic to call you back so you could make that appointment. So you could say that that perhaps this wait time represented in those in those red dotted boxes could perhaps be driving that drop off later in the process. Now, looking at the customer experience audit findings, we see a similar kind of pattern where we have here the average customer experience audit scores for each of those phases.

So those scores are out of out of ten. And you can see where where the lowest score is very difficult and the high school is easy. So you can say that those early stages are the most difficult and where those early stages are difficult, that could be perhaps driving the drop off. Another interesting finding was that cancelling or rescheduling an appointment was was very difficult.

So if someone did want to reschedule appointment an appointment because they weren't able to make it, then because it's really difficult, perhaps they just wouldn't follow through, follow through with doing that. So with those results, we or the team looked at it, looked at it, looked at the results to prioritise the sludgy points in the journey and came up with a couple of areas to focus on.

The first was that it took around a week to book an appointment, so that, you know, you can save with that wait time that could take up to a week, which is of course, you know, very sludgy and could be driving. The drop off was also very difficult to find information about the service. So very difficult to find information online about where to book and what the service was.

And there weren't any reminders. So it was very easy. Once you'd booked the first appointment and attended the first appointment, if you hadn't then booked appointments thereafter, you may forget that this service even existed. And you can imagine, as you know, if you're a new parent, there are lots of other things that you need to remember. So without those reminders, there's not much hope.

So with that sludge audit completed and the priority areas identified, the team attended a two day solution building workshop, which was part of the sludge-a-thon. And here they developed five prototypes and this was just done in the two day period. Kate will go into detail about how they develop these prototypes, but the initial prototypes were a reminder sticker on the Blue Book, a script for nurses to encourage carers to book additional checks.

So once they'd attended the first cheque book and book in the next one, a behaviourally informed SMS reminder to two carers to read about the appointment improvements to the website. So there was more consistent information and also a magnet which had a checklist on it and illustrated progress towards a final checks that kind of harnessing the goal gradient effect there.

So with the sludge-a-thon completed, the team took these prototypes and developed them. So I'll pass over to Kate now he'll talk you through how how they did that. Thanks Alex. So to dig in a little bit further, we did a whole lot of research side by side with the sludge-a-thon with parents. And what we found is that all parents and carers are not equal and in their attitudes and how they understand the health checks.

So we found that was sort of four main types of parents, one who just need to be informed about a health check and they were happy to take action and book themselves in the next type of parent they actually needed, prompting to remind them that it was time to book and a time to attend the check. Our third pay person was the experts who said, I already know about health checks.

Why do I need to do this? And they needed some more convincing that actually this was really an important part of checking in with their kid. And it was done by experts. And the last, which was our most vulnerable cohort, that they were very afraid about what they might find in a health check, how they might be seen as a parent if a problem emerged and they were generally fairly overwhelmed and had a whole raft of challenges they were facing.

So we took these four mindsets. And when we designed the solutions, we really kept them in mind to make sure the needs of these four different types of participants were kept in mind. So with all those great we actually developed with the nurses and with parents three different options to test with parents. And you can see here the magnet and some of the things that we tested.

So we did three rounds of testing with all of the resources to find out what really worked and some really surprising things came out. So you could even say from these three that you can say one's in a government blue colour and parents reacted very strongly and said that was scary and they wouldn't book a check. It was just too off putting and very simple things like we tried circles for ticking off your appointment versus squares and everyone said squares they'd be happy to tick off as they attended, but circles they wouldn't use them, which we never thought would make such a difference.

We looked at the imagery and apparently the photo, the pictures of the child getting older was important, particularly for low literacy, where they understood that the checks happened across that expansion to five. So there was some great things. The other challenge we use quite a lot of behavioural prompting and we really drew on some of the great research on what works at outpatient appointments.

But we tested these out and what came out is some of the more strongly worded reminders and behavioural prompts are parents really rejected quite strongly and it interfered in that really positive, trusting relationship with the nurse. So we had to really adjust how we prompted people to turn for an appointment. The other thing we really looked at is across the journey, where was the key points that we could make sure we checked in with people?

So the Blue Book sticker was actually added when parents were and carers were in hospital. It really set up for success and expecting what they could do with a health check. And we knew because they were in hospital with their blue book that everyone would have that added in. So it didn't take any active actions from people at the 1 to 4 week check.

We developed a script with nurses and we also introduce proactive booking. So this script was vital in talking parents through why this was important and then inviting them to book in the next appointment and the drop off between the 1 to 4 week check and the 6 to 8 week check was the biggest drop and required them then to attend an appointment in a clinic rather than the nurse coming to them at the check.

We gave them this magnet which people put on their fridge, very visible.

They are

Hugely appreciated. Such a simple solution, but the design really made all the difference. We made some big changes to the website, making it easy, accessible and checking with parents that the information really resonated with them and address their key problems. First and last of all, once they were booked in, we increased the SMS message, reminding so that people didn't forget in their busy and overwhelmed schedules to turn up for their appointment.

So the results, well, we were, I think Alex very pleasantly surprised is an understatement. So we had hoped and we thought was quite ambitious, that we would increase the booking rate by 10%, but we actually increased the booking and attendance at health and development checks by 24% and if we took into account the change in the birth rate, it was actually 30%.

So you can imagine how many more kids were getting these vital checks. I think the success rate of the script was also a key thing. 97% of nurses actually said they found the script really helpful, while many of them were highly experienced nurses, really having a systematic way to talk about what they did, some of the things they said is we knew what we did, but we've never explained to a parent so clearly how important it is and what our expertise is.

And 89% of parents took up proactive booking. We thought maybe 25%, but a huge number said yes, this is really important, we're going to turn up, we're going to do this. And they did in fact go on to attend those appointments. So that's our big, big finale in our results. But it was an amazing outcome and has been introduced in the Illawarra now does this as routine.

Back to you. Alex. Thanks Kate. What a fantastic outcome. And I think what that tells us is that such audits are really valuable in this space. Obviously as well as that the the great work that Kate and her team did refining those results and use it for finding those solutions and user testing them. But it also tells us that such audits provide this systematic method for understanding and supporting service improvements.

And we've seen that that across a number of stage audits that we've done, some of those are up here. So things like applying for a companion card and improving the quality of applications for early childhood disability funding, these are some of the areas where we've tended to be really successful. So what we've what we've done as well is built a number of resources so other teams can bust such.

So this is our sludge busting tool kit which you can access on our, on our website at the sludge tool kit. The URL is on the screen there. But if you just Google it, I'm sure it will show up. So we have our sludge guides, which are channel based guides to reduce sludge will also be releasing in a couple of months, have sludge scales and our sludge method guide the sludge scales at the scales that we use to complete the customer experience audit and the Sludge Method guide is a step by step guide.

I suppose for our method so that you can see what to do at each step. We also have sludge audit templates, which you can use to do things like interviews and journey mapping for your sludge audit so how you can buy sludge. So firstly, access our sludge guides and share your feedback. You can also the such toolkit is available via that QR code and you can also tailor your own sludge busting methods.

So the New South Wales method is very much developed for a New South Wales context using the New South Wales Government customer commitments. But we'd really like to see the method taken up by other jurisdictions and we and we have seen some of this and we've seen the way that it's been tailored and adapted to different contexts, which has been really interesting.

So we encourage you to do that as well and we'd like you to start your own sludge audit, you know, using, using your method, and we'd love to hear more about it. So please email us at Sludge at customer service Dot NSW Drug Coverage. Thank you. And thank you too to Kate for the excellent partnership. It's been a pleasure.

Likewise. It's amazing what you can do when you do things together.

Thanks, Alex and Kate. That was a great presentation and great like wonderful results. Congratulations, but also great to see those resources publicly available to you. Just a quick reminder before we move on to our final presentation, you can add questions into the Q&A as we go and we'll be curating those in the back end. And then we'll move to a Q&A session with all of the panel members after our final presentation, which is from Professor Liam Smith from Behaviour WorksAustralia at Monash University.

So Liam is the director of BehaviourWorks Australia, which is based in Monash University Sustainable Development Institute. Since establishing behaviour Works Australia in 2011, Liam has overseen significant growth and the research group now has a portfolio of over 600 behaviour change projects in collaboration with government and industry and a staff of over 30 people. Liam has published numerous research papers, research reports and public discussion paper pieces and is an active contributor on the Board of Inclusive Australia, the Biodiversity Council, as lead counsellor and South East Water's Customer Engagement Council and Monash Sustainable Development Institute's Executive.

That is an absolute mouthful. Liam, congratulations and welcome. I'll now pass over to you. Thanks.

Thanks. Can you hear me again?

Thank you.

And just checking that you can see my screen.

We can also see your slides. Thanks.

Yeah. Thank you. Excellent. So I'm just going to start by acknowledging the traditional owners of the land. I'm coming to you from today. For me, that's there Wurundjeri people of the nation in the eastern part of Melbourne. I'm on land that is very beautiful. It was never ceded and I pay my respects to elders past, present and emerging.

I'm going to talk to you today about using behavioural sciences to foster systemic change. And and really this this presentation comes about over the last 13 or so years that we've been that behaviour has been running and we've been thinking about the work we do and the impact it's having and whether that impact could be could be greater or indeed is, is potentially making problems worse.

And I'll explain that in a minute. For those who don't know, behaviour works, we're applied research unit. We have about as as introduced 30 staff and we work on a lot in applied projects. So much like many of you. And I guess reflecting on some of that work, we we got the feeling that not only were we, we weren't always making the difference we wanted to make.

So I'm going to give you two examples. One one, we were involved in, one we weren't. But but there are two examples of what you'd probably think of as outstanding successes in behavioural science. And the first comes from from Australia in the in the in the city of Townsville in Queensland where the issue at hand there was well the task at hand was to try and reduce energy consumption in households.

Those of you who've been involved in any energy consumption projects will know there's actually a lot of behaviours that you can target in in energy consumption and in fact that project and some of the work that happened around it listed all those behaviours and came up with about 240 different things that householders can do to help reduce energy in their home.

Doing that exercise, it was helpful, but what was even more helpful was ranking them and they use two criteria impact and likelihood. So the difference those behaviours would make as well as the likelihood people would do them if asked and to cut to the chase, one of the most impactful and likely behaviours was for householders to paint their rooves white, So to to change the colour of the roof or indeed for new builds to have white roofs installed instead of darker colours.

So my first slide is an aerial view of Townsville, a new estate here on the left, which you can see has lots of white roofs. And then on the right is an older estate. But this is a place where the lot of the roof colour has changed over time. Not many dark rooves there at all. The reason why this looks successful and is successful was because the Townsville City Council took a lot of time to try and change behaviour.

They did some research to understand drivers and barriers. They made the behaviours easy as possible and they did and they incentivised that and all those right things that you should do to get it to happen. And so it's terrific and a great success. And, and while more needs to be done, it is nonetheless success. The second example I want to talk to comes from South Africa, and many of you might be aware of the campaign called Day Zero.

This campaign was run because effectively their dams storages were running dry and really running dry. Day zero was given was was was given to the time and dates, which was the 3rd of July 20, 19 six. But sometime between six and 8 p.m. when the water would run out and there would be no more water left. And so this was this was a really compelling campaign.

It was terrific in getting water consumption reduction. I think that went down to 50 litres a day that didn't install the nets that was was down to 25 litres per person per day here in Australia. We thought we were pretty good around 150 during our drought, so significantly lower than that. Again, fantastic. And arguably a huge success story.

I think there's a couple of issues with these to these two and I think many other behaviour change projects that are run, which is that it's effectively they did get the behaviour change, it rained and they've gone back to where they were admittedly a they're built in some desalination plants a bit like we've done a lot here in Australia in other parts and, and that desal is now been supply, which means essentially they can continue doing what they've always done, turn on the tap and then it goes out to a drain somewhere and disappears.

Yeah, some gets traded, some gets reused, but most doesn't. In fact what we need is really significant systemic change to the water system. The water people tell me there are seven different sources of water and we need to think about how we use every one of those really well. And and so in some ways, getting this behaviour change kind of said, well, let's just stick with our take use waste sort of approach.

Likewise in energy. Sure, we got a great outcome in Townsville. We know though that households, the biggest of energy use in households are the way they face the size of them and their insulation levels. And so in some ways we sort of little bit fixing the problem, but we're kind of saying we've done really well, but actually households are still nowhere near zero and need a lot more and have a lot more to do.

And indeed the whole grid needs to be greened to be much better from a carbon perspective. So these examples and there are many others and most of these examples, by the way that I presented, I come from the environment area and probably that's because the systems in environment are probably most flawed. It's certainly flawed in other areas, but they they are reflective this idea that agency reinforces structure and structure reinforces agency.

And what I mean by that is most of the behaviour we do is driven by one of two things, or at least according to this theory. There, as many of you know, there are many other theories that say many different drivers of behaviour, but from a sociological perspective, behaviour is driven by the structures. So that's the physical infrastructure, but it's also the policies and the laws and the norms that exist around us and an agency which is our ability to act as free agents and make choices.

And so my concern to some degree with these two examples and others that we've worked on was that if we give people agency and they can act with agency as in they have the freedom to act and choose. Then it's saying that nothing is wrong with the structure and the structure doesn't need change. So and then that reinforces the fact that we have agency.

So it got me worried that what we were doing was reaffirming existing structures when actually we wanted structures to change. So that got us thinking about, well, could behavioural science be used for better change or more structural or more significant change? And it sort of took us to sort of three main points I think that I want to make across the course of this talk.

And the first is we should choose behaviours that lead important change, so changes that, that make a difference. And, and I think when we think about behaviours again, this audience knows really well, but there's, there's this audience and then this behaviour and so on. The first part of that there are some particular audiences that we really may want to focus on.

Leaders I think in particular and the decisions they make, we know lead to all sorts of implications down the track and, and, and we've started doing some research on understanding leadership decision making, and I think more needs to be done and they need to be a target audience that we concentrate on more in the behavioural sciences to really try and understand why and how they make decisions and then how we can potentially influence them.

And of course the example on the right, you know, there are still behaviours that are just genuinely very impactful and this example is of around a river and the area right around it called the riparian zone, which obviously if protected by farmers, does a lot to prevent the river silting up. So their particular audience that their decisions around what they do in that one space can make a big difference.

And the other sort of audience I think is is interesting is is when you think and learn about systems and transitions theory one of the hardest things is to decommission behaviour or decommission the existing structures or processes or procedures or laws or whatever they may be starting the news. The easy part, decommissioning the old is the hard part.

So that got us thinking that maybe their behaviours like for example in the forestry industry and here in Victoria we've just declared a ban on native forest logging. You know, one of the things we need to do to enable that is find new jobs for people that we're working on in the forestry industry. And so if we can focus on that particular audience and behaviours that are enabled, all them gainful employment such as, you know, revegetation work or got tours guiding or all sorts of things, then those sorts of ideas should be a priority because they help enable systemic change.

And I guess the third group of people that are worth considering in particular are key cogs in the wheel. I'm sure many of you have run projects that have been successful and then struggled at scale up stage. So instead of, you know, we've got these great results and we think we can make a difference, well, and then and then for some reason there's a particular act or a particular system or process that doesn't change and it doesn't get scaled as as we'd like.

So it struck me that the behaviour of some of these key cogs in the wheel managers or decision makers, perhaps it is the decision makers again, but also others could well be really important and we should think about their behaviour and how we might help them act on or enable or implement really successful behaviour change interventions that we develop.

And of course there are the behaviours themselves. So three audiences I've spoken about, but then there's the other, the other side of that is what they do. Some of you be familiar with this. I talked about it briefly in terms of the Townsville project, you know, you might want to prioritise behaviours that are effective and easy. Sorry, I don't know why that went back…. but, but increasingly I think he is particularly an impact is changing a little bit.

Some other co-authors from University of Queensland, I have a paper under review at the moment where we surveyed 140 experts about the impact of a particular behaviour out of different particular behaviours and the ones that have come up a lot more and more are around lobbying and voting. And I think lobbying is particularly a space where we don't tend to go and government doesn't tend to go so much.

But as behavioural scientists we really ought to think about it. It reminded me of a project again we're involved in and probably many of you are familiar with, about a campaign that was run by Victoria to try and encourage companies that were importing palm oil to switch their import to importing to sustain a certified sustainable palm oil. When you went through the exhibit at the zoo, you got to it got the opportunity right there and then to send an email to the five CEOs of the five companies saying, please do not import CSP anymore.

Sorry, please do import CSP, don't import unsustainable palm oil anymore. And and that led to pretty significant pressure on those companies. And eventually all five of them have now committed and I think actually most have now transitioned to only sustainable palm oil. Of course, consumption and particular consumption behaviours can be really influential on systems as well. And just to give you a little example, we were involved in a project with Taronga Zoo which is trying to encourage people to purchase MSC Seafood on Marine Stewardship Council, Seafood and and while while this project in and of itself was reasonably successful, the more important story out of this was the conversation that happened between the this is

Taronga Zoo and and Woolworths who realised that they were going to be telling about 2000 visitors a day to a seal show to buy MSC products and they didn't stock any. And so what we saw really quickly was not only a change in their supply so that Woolworths and then and Coles as well did start supplying the certified seafood products, but also saw a commitment to help supporting the research and funding to try and get, get, get more to see what works and really encourage this program further.

So the first one, so just to recap, so that's just choosing behaviours that we think will make a difference, whether it's from audiences or particular types of behaviours. And so the second one and the second why I guess hat to wear is a behavioural science, just when you're thinking about systems is to think about most people will know that term.

I assume on this, on this call, which is about one behaviour leading to another and essentially this spill-over is that by doing one behaviour predisposes to do something else. And in this kind of line and this is an area where research is just coming on in leaps and bounds, I think, you know, spill-over probably first mentioned maybe ten or 15 years ago maybe and in the literature, or at least it became a thing in the literature then.

And since then, I think there's been like many PhDs, lots of research happening. And as far as I can tell, there seems to be some key, key things that help people transition from one to another. The first is how they see themselves. So, you know, if I see myself as a as a particular type of person, then that predisposes me.

To do another behaviour. If I think I'm a can do person with self, with self-efficacy, it might lead me. And of course if I have a different attitude to something then it might help me shuffle through to another behaviour. So these are some sort of key key factors that have been identified. There are probably others that are around, but they seem pretty important.

And just to give you an example of what interventions might look like, you know, like if someone's on the train and you say to someone, Hey, by catching the train, you're taking a commitment to climate change action, or in this or by catching the train you shine, you could do something to address climate change. Then these sorts of messages are applying to self-efficacy in this case and and perhaps identity in the first one and maybe identity here as well.

So there's a sort of way that we might be able to foster spill-over to get to another potentially more important behaviour. I guess the question though is what's the second behaviour? We ought to encourage? And you know, some of the work we did a while ago, this is certainly Ben's work, you know, this is an impact likelihood matrix of all different behaviours around water savings.

And what the work she did was to say, well let's look at how people see them are similar or not. And she found these groups that sort of some are more similar than others and that's, that's worth exploring. Sure. Try and work out which are more similar behaviours but actually and sort of leverage off these low impact, high likelihood behaviours and then push up to more high impact behaviours in the top quadrants.

So that's certainly one way of thinking about what the second types of behaviour might be. So similar ones that are more impactful certainly would be one way to think about it, but actually I really like the idea of thinking about leverage into policy support. Those of you who know Stern's work about policy support as being a behaviour technically is a behaviour start, so it probably isn't.

But that's a right and, and I think policy support is a space where I think there is some more opportunity and I draw on some work done by Caroline Noblet as part of a Ph.D. with John Ferguson probably ten years ago now, where they looked at support for wind farm policy. And what they found was that people were doing more in the environment space in home, so people that were turning off the tap and the brush their teeth or saving energy or water or waste in whatever way they can were more supportive of wind policy.

Now that that's no surprise to anyone, that's sort of a bit of a no brainer. But the interesting thing was that they controlled for attitude. In other words, people to people who cared the same about environmental attitude. So two people who cared the same about the environment, one supported wind farms more than the other. Why? Possibly And it's not it's not a set design, but possibly because they were doing more at home.

In other words, by doing something, it might change something inside us that might lead to more symbolic support for policy. And I think if anything, when I think about behaviour and system change, we really ought to have our eye on policy and policy support as one of the, if not the ultimate goal. And the last thing I want to talk to you about, the third way that a behaviour scientist can wear a hat to think about systems and this isn't sort of related to being active an actor in trying to get changed, but more in trying to understand the systems comes from this idea that we can use behaviour change attempts to act like a

radioactive dye to diagnose the problems. So, you know, if you think about a typical intervention, we might design, we send out a letter or we restructure an environment and we get sort of 20% more people doing something. And that's that's great, right? And we say, yeah, with minimum cost, we were able to get quite significant changes in behaviour, but you use that same example and to take this kind of approach would say, okay, that's good and that's that and we're happy about that.

But 80% of people didn't change and I really wonder why. And so that kind of thinking leads us down the path of saying, can we use those 80% to try and tell us how the system could change or how it could work in order to better enable our behaviour? And so I'm going to use the example here and there's a couple, another one that a doctoral student was working on as well in e-waste, but I'll use this example that a colleague of mine, Rob Raven, is using in low waste living.

So there's a CRC, a corporate research centre for low waste living, and they found about a dozen people that were really passionate about waste. They really wanted to change. So they they were motivated, right? And they signed up to a program of low waste living. They were given lots of things they could do to help reduce their waste.

And and then they were sent out and given support, you know, etc., to try and get changes to happen. What they found at the end of that project was that actually most people hadn't changed their behaviour. So these were the motivated people that really wanted to, you know, cut down on their food waste or cut down on their on their packaging.

And I said to them, why didn't you try it? You said you were motivated and you really wanted to. And they gave some good answers. So they said things like and not only they gave some good answers, they gave some good solutions. And and if you look at the sorts of things that they suggesting, so many of them are regulatory or policy based, they're the things that we need support for in order to, you know, mandate, end of life producers, etc., etc..

So they were saying this is where the system pushes back on the motivated person. A lot of the interventions we work up, we do in behavioural science work on motivation, but by taking those that are already motivated and getting them to try and get change and seeing where it pushes back is one way of sort of understanding a system to try and work out how we might change it for the better.

Like I say, we're also doing another project in this space in e-waste, and you can imagine people who are really keen, who have got an old toaster and a whole kettle to do the right thing by that, to work out what why they cannot can't do something apart from put it in in their waste bin, which is obviously not desirable at all.

So just to recap, I started with some examples of behaviour sites trying to change behaviour that had been quite successful, but yet, you know, it still got me thinking the energy system really hasn't changed or the water system really hasn't changed. And, and you probably noticed a lot of these examples are from the environment and that's probably where there is more systemic changes needed.

And also this is an adaptation of a presentation I gave to an environment conference. So to some extent it was a bit easier to keep those examples in, but you can probably say it's a similar things in in the work you do. And I think a key question to ask yourself is too, is the system roughly right or do we need more wholesale holistic change to this system?

And if the answer is yes to that set to that question, then I think some of these approaches might be helpful in the behavioural sciences has something to offer. So the first is to identify target and actors that make a real difference or have a greater impact than just the behaviour. Second is to investigate and foster spill-over wherever you can.

So to try and get people to go from one behaviour to the next to the next to the next, ideally with policy port support as potentially an end goal because that will enable the policymakers to do their work and finally use behaviour change to test with the motivated to test with systems. Push back. On that note, I will hand one think I'm just about a minute under and I think my move to the session on questions.

Thanks.

Thanks very much Liam. That was great. So that does bring us to the end of our presentations for today's session and we can open up to a broad Q&A. If you are holding on to a question, please submit it through the Q&A function. We do have a couple of questions that have already been sent through. I'd also like to invite presenters.

If you have a question for another presenter, feel free to pop that into the Q&A function as well. Okay. So Nick, I might start with you, given you've had the longest pause from the microphone so far, I guess a fairly broad question. What do you think the implications for this research on the way that governments communicate about inflation more generally?

Yes. So no. Good question. I think a couple of things. One is that the we kind of assume that people have a better sense of inflation, both the bounds and and kind of what the future is than than we might might actually kind of fit reality. So I it's partly how to, you know, reinforce messages around what's what price changes are and also to kind of better understand which groups might have what might be called kind of irrational, kind of expectations or expectations of inflation, which might not line up with with kind of realistic change.

I think there was a there was a question from Hanne, which I think kind of touches on on this point, which is around kind of what people really expect, what people mean or think about when you kind of talk about inflation. So I think that's there's some work we're doing for our next survey, which is, you know, trying to get a sense of how people understand a kind of probabilities or percentages and how that relates to a kind of real world kind of measures like like inflation or other things.

So I think there's a there is definitely some work around it. It's giving people the tools to translate quite complex concepts into things which have meaning for them. And I think you can kind of see that with with our data, which is where not surprisingly, I guess kind of education is not only predictive of expectations, which are kind of less likely to, to line up with reality, but also where people with relatively low levels of education are more affected by kind of cues or priming.

So so part of it is kind of to give out those differences. I think also the the way in which could reinforcement of of kind of messages and kind of making it's kind of a message really consistent about what what inflation is, what it's likely to be in the future and otherwise. I think people are kind of making decisions which which may not line up terribly closely with with reality.

And and that's kind of reinforcing inflation. We kind of see that where people's kind of at times of as a kind of part of the presentation at times of uncertainty or times of change, people could fall back on on quite kind of broad kind of measures. And and that's likely to feed into own decision. So I don't know, I think I kind of talked around your your question rather than directly addressing your question.

But I think the other thing is, is I guess a broader point about not only do we really careful about what information we use to make decisions. So I think there's there's a lot of reliance on a kind of inflation expectations, data from the RBA and and others in economic policymaking. And I think that's okay that there's some information in there, but also being really careful about where that information is coming from and this kind of comes back to, I guess, a fair bit of the work which we do with the Centre for Social Research Methods, which is kind of trying to collect as robust a survey information as possible and really kind of interrogating

how accurate the surveys are, which we rely on in terms of kind of making policy and healthy informed decisions. And it's very easy to, like I say, every month or so when the Melbourne Institute puts out their inflation expectations expectations data, it's interesting and and it's kind of it has some information, but it's that reminder of like, where's that coming from?

What's the sample? What information that people been given when they're giving that information in order to know how how much kind of how much reliability there is and how much we can use that data to to to kind of inform decision making. So, yes, it's partly about also really getting interrogating and understanding where our surveys come from, what information people are giving, and how is that affecting the responses they're given, given how much government relies on a kind of survey of public opinion data across a range of topics like just inflation?

Thanks to me. Well, I had like three other questions to ask you, but I feel like you've answered all of them in that very fulsome...

Or none of the I kind of answered all of them and not that one response.

Thank you. That was great. I'm Kate and Alex. I might wave on to you now. There's a lot of interest in the the sludge audit methodology. And that was great example. Thank you. There is one question here. We're wondering if you've found that the audit has been taken on by other teams that you've worked with. So this was a great example, I guess, of a partnership with New South Wales being you and another Entity.

Has there been any resistance that you've encountered in putting forward an idea to do a sludge audit and had a team sort of push back on that?

Yeah, that's a really good question because I guess the term audit could be perceived as a little bit of an intimidating thing where we come in and tell people they've done something wrong, which is of course not how it's intended. So I think the way that we've approached developing the method and promoting it to other teams has been very much framed as a tool for teams to help achieve their goals.

So we know that teams in New South Wales Government are all working to improve the services that citizens are accessing. So really we've positioned a sludge audit not as something that way as the behavioural insights Unit come in and do all them to tell them, as I said, that they've done something wrong, but really as a tool for those teams to use to achieve their goals.

And we've kind of built this up across a number of years. As I said, we started in 2020 with some pilots and then we had two years where we ran sludge-a-thons, which were a call for expressions of interest for teams to suggest processes that they wanted to audit. So it was a very ground up approach where people actually put forward that they wanted to audit as opposed to us suggesting them.

So this then built more awareness of the study audit method across government, and we had a range of different teams who, who then participated in sludge-a-thon. So it was, you know, teams like Kate's who specialise in customer experience. But then it was also teams who owned the product. So for example, that was a team who participated last year who were from Transport for New South Wales and looked after licences, so looked at the bus driver licence process and so they went necessarily specialists in customer experience or behavioural insights.

They just wanted to improve that process. So it's been really good to see it taken up by different types of teams. And now something to your point about how it's been taken up, we have our online sludge audit tool, which is a web application for people in New South Wales or teams in New South Wales Government to audit their processes so we can see when how that tool is being used and we're just constantly surprised by new audits being started that we didn't where we haven't even engaged with with the teams.

So they've just heard that there's this tool available, they use it, they can plate an audit and use that audit to make improvements. So that's been really encouraging, surprising and really exciting to say, that's great.

Congratulations. And there are a couple of questions about the sludge-a-thon. So there's a question here regarding the post slide just on changes. Are you tracking them to say that they're sustained long term?

I can speak for our project. So yes, we do ongoing tracking. So we track the number of developmental checks that children are attending across all eight checks. We also track this QR codes to link to the website. So we track the different methods. So we look at that sticker on the blue book when people are in hospital, but also the magnet to look at the take up and utilisation of that and on the website itself.

And we also track failure to attend rates. So the team then draws identifies quality improvement opportunities and I think the results really pointed to that SMS reminder. There was a lot of restrictions on when that could be sent out and they're already doing some work to get more flexibility and trial the SMS at different times.

Great. Thanks, Kate. Alex, did you have anything to add or should we move on to another question?

And no, I mean, as as you can see with the work that Kate's done, it's very much that after the teams have completed a sludge audit, they can either track the impact of their solutions by looking at different behavioural outcomes that Kate's team has, or they can complete follow up audit. So that's something that we encourage as well.

But yeah, that, you know, it's been really wonderful that that Kate's team has really taken it and run with it and continued to track that impact.

Thank you. And we've got another question on the slide just on events. Could you tell us a little bit more about it and the outcomes that it had?

Yeah, sure. So we've had to strengthen so far. So it's a large audit. I mean, but with sludge audits that can be conducted at any time. The sludge-a-thon's were a way to drum up some interest. So the process is that we put a call for to welcome expressions of interest from teams across New South Wales government.

And the idea is that they put forward a process that they think is sludgy, that they would like to that they would like to improve. So last year with our sludge it's on, we had around 50 or over 50 expressions of interest and then we we selected a group of 12 teams to participate. And what those teams then do is attend a masterclass where they learn how to complete a sludge audit.

So that's a a couple of hours. But they they take in through in detail how to complete a sludge audit. And they then have around five weeks to audit the process that they've so it's a structured five week schedule where we we put in milestones as to what they should achieve each week. So for example, with the first week they should have done their behavioural journey map, so they work through that throughout that period.

Each of the 12 teams has a dedicated mentor from my team, the Behavioural Insights Unit, to support them through that. Once all of the teams have completed their audit in that five weeks, there's a two day solution building workshop where they buy, they develop their prototypes, conduct some some proxy user testing as well, and also present what they've what they've developed to the wider group.

And then from there that they go on and implement solutions. And we've had some really great stories of impact that have come out of that. Obviously, Kate's team, which has been wonderful, as well as teams who've completed follow up sludge audit. So, for example, the bus driver team that I mentioned earlier, they reduced the time it takes for someone to get their bus driver's license by an hour.

The team who looked at the process IT process of applying for a companion card also reduced the time it takes by an hour. And yeah, so there's been some really great outcomes that have come out of those.

Great. Thanks, Alex and Kate. We've just had a couple of questions come in for you. There's a few in there, so I'll try and break them down. Can you tell us what plans there are to scale this up at other local health districts?

Great. I'm all about the scale up. That's one of my passions. So We've been working with New South Wales Health to try and expand these solutions across the state. One of the challenges is every local health statistic district is very different. But I think the strengths of this was the sludge audit really gave a really systematic way to describe what was happening and to cost it, which helped with making a business case and the different ideas to make the business case.

And really the results that we've got are being very convincing. So we're working with New South Wales Health and I think a few districts will take it up. Initially and hopefully, yes, we'll see more of this state-wide.

Right. Thank you. As a follow up, I'm interested to know whether you had a particular cohort in mind that you were hoping to target to increase access to child health checks.

We really looked across the board, but I think vulnerable cohorts really came out as a particular target audience. They were the ones that struggled most to understand what was on the website. They were the most overwhelmed in doing this in in kind of booking, etc.. So we did lots of when we looked at the process and also particularly in that really testing out the solutions, we worked with one of the Aboriginal community health centres to really understand their experience and where their sludgy bits were in particular, and particularly some of that emotional and psychological reaction because of the they personal experience and the historical context.

So that was really important. So it was a for everyone solution, but making sure that everyone was brought along as well.

Okay. One last follow up to that. Did you get any data around particular cohorts that formed part of the increase of uptake in the making of appointments?

That's actually a really good idea. We didn't really look specifically because we used de-identified data about who were the people who were really taking it up. But yeah, that's a great suggestion. I'm not sure we will have access to that information, but that would certainly be interesting. Okay.

Great. Thanks so much, Kate and Alex. I'll let you have a breather and I'll move on to you later. Liam, I can see you’re answering questions in the chat. Thank you. But for the benefit of people who aren't monitoring the chat, we have had a question about the upcoming e-waste pilot, how people can get involved in.

Oh, so the short answer is it's a doctor student of mine who's just coming up for confirmation. So she's about a year in, just under a year in. So I think she'll be trialling it next year. I think I'll put a name in the chat. Her name's Lauren Brumley. I'm sure anyone interested can contact her directly, knows literature really well, so will be able to tell you what's already known about e-waste.

And there may be opportunities to collaborate with with her and I guess myself, one of her supervisors into next year when she starts to take ideas into the real world.

Great. Thank you. And I'm just going to repeat another question that I can see you've already given a quick answer to. With interventions to influence policy support and lobbying like behaviours are tricky to get up from within government. Do you have any suggestions about how to present pitches for these sorts of projects in a way that is politically viable from within government?

Yeah. Great question from Nick. I wouldn't expect any less. So I think I've written a couple of responses. So perhaps just reiterate, I did present several examples from those that are government owned. So Taronga Zoo is government owned, zoos, Victoria is government owned and and they've been quite progressive I guess, in trying to advocate for not necessarily party support but certainly policy support and and particularly small pay policies are the example of, of the, the sustainable palm oil, which is certainly a small pay policy, it's a business policy or a company policy that they were kind of trying to trying to lobby and get get changed to.

So And so in order to do that, I think a strong mission and an IT and a CEO who's who is pretty tough. I think. I think you need that. I know that the CEO of both organisations, particularly zoos Vic has been hold over the coals a few times for stuff they've done but is just absolutely mission focussed and oriented.

I think it's true of a lot of government bodies that are agencies that are not caught in core. I think you probably there's a stronger appetite for change and for implementation. We've certainly done some work with API's around the country that is a bit bit more progressive. So perhaps just structurally there are some that are just one step removed from the central of government that might be a little bit more keen.

I think there's there's also the example that I gave where government came to us and said we want the policy support. That was that example of transport network pricing, which is this idea that you pay more for travel depending on where, when and how far you travel. And in things like scrapping your registration fees and all those sorts of things in exchange for this sort of which is pretty significant policy change.

And and how can we engage decision makers in the public in trying to get more support for that? So it was existing policy that I were trying to get strength for. And and so that was kind of what that was one of the projects we looked actually, look, decision makers, we had 20 former politicians and I'm senior advisors and I'm chief of staff.

I'm coming to a room under Chatham House rules to discuss the policy and and how it would get through, what their different perspectives would be. And and it was it was fascinating to say the least. But but really it was it was it was trying to help a policy get up and that from a government perspective. So I think they had kind of a couple of angles.

If there's already policy that they want support for, then obviously they're keen And and we're they're very mission oriented. And perhaps one rebel or away from government from the core government might be a bit easier.

Yeah, great suggestion. Thank you. And I guess just a broader question for you, Liam. Is there anything that you can share that you and your team have developed or had plans to develop in the sense of like tools or materials or resources to help other behavioural scientists think more about system level changes.

And not yet look, you know, I think this is this is this kind of ideas have been brewing for a few years now that I presented today. I presented them earlier this year and I know that others, Michael Hallsworth and others are writing about these sorts of nexus between systems and behaviour now. And, and so I think we're just discovering that space at the moment.

You know, some of my suggestions were kind of tools like when you think about behaviour, think about what's the ultimate policy that we could support or can we leverage off existing behaviour, try and get there at maybe an impact. Likelihood metrics helps you do that. But, but yeah, look, I think, I think we're still early stages. You know, our institute also is, is, is really pushing our strategy.

We have systems and behaviour in a circle and, and there's, you know, my three doctoral students that are looking at that at the moment. So we're probably a couple of years out before we share a lot, but we've probably been in most of the same spaces that people on this call have been on for much of our journey as well.

So, you know, it's essentially it's one of this is a call out to all of us to create tools and ways of thinking, I think, to try and overcome the criticism, which has been around for a long time, that, you know, we'd kind of tinkering while the big problem just sort of rolls on. And I think that's been there a long time right from the start of the unit in the UK, really.

But but yeah, I think it's it'd be great if we could all put I think he gets on to try and come up with ways to, to, to do it better.

That's great. Thank you Liam. And on that inspirational note, we might wrap up. I think we've exhausted all of the questions in the Q&A channel and we can give everyone back 18 or 19 minutes of your morning. And before we do wrap up, there was a question around the recordings being available and the slides being available. We will publish recordings of these sessions on our website at the end of this of the series.

So the last session is on the 30th of November, so probably early December. And we'll just double check with all of the presenters that they're happy for us to share copies of their slides as well. A big thank you to our presenters today, Nick, Alex, Kate and Liam. We rely on fantastic pieces of research and your willingness to share, present them for BI Connect to be a success.

So thank you for volunteering to be part of today's session. Thank you to everyone who has joined us online today. We've had just over 150 people log into these sessions, so that's a great sign of the interest there is in the content that was shared today. I also reminder this is the first of three sessions for BI Connect.

There are two more to come. The next will be held on Monday, the 20th of November, from 2:00 onwards. And the topic for this next session is being at work, building safe and inclusive workplaces. So we have Liz Convery from the Behavioural Insights Team of Australia presenting our own Nick Hilderson will be presenting on some really interesting machine learning analysis in the cybersecurity job ad space and also Dr. David Smerdon from the University of Queensland.

So thank you everyone. Hope you have a great day and we look forward to seeing you at the next session, next week.